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This paper reports on an ongoing project aimed at developing an inter-institutional system of 
professional support for the improvement of the Geometry for Teachers (GeT) courses that 
mathematics departments teach to preservice secondary teachers. In alignment with the 
literature on improvement science (see Bryk et al., 2015; Lewis, 2015), it is essential to develop 
and deploy practical measurement tools to inform improvement. We describe three key forms of 
measurement our team has been using to drive this work as well as some preliminary findings. 
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Introduction 

This paper reports on the development and use of instruments to study instruction at the 
college level, specifically focused on the undergraduate geometry course offered by many 
university mathematics departments for pre-service teachers (Geometry for Teachers, or GeT 
hereafter). The teaching of geometry in schools has been identified as weak and resulting in 
unacceptable levels of student performance (Clements, 2003; NCES, 2012). High school leaders 
have often described it as hard to find high school teachers who want to, and can, teach 
geometry. Thus, we consider increasing instructional capacity for high school geometry a 
systemic problem. We pose that the university GeT course is a crucial lever in preparing pre- 
service teachers to teach high school geometry, particularly as one of the few college 
mathematics courses that directly connects to high school geometry content. Yet, many 
mathematics education scholars (Zazkis & Leikin, 2010) and mathematicians (Wu, 2011) have 
questioned whether the content of university courses is sufficiently connected to what secondary 
teachers need to do their work. This led us to investigate the connection between participation in 
GeT courses and increased mathematical knowledge for teaching geometry (MKT-G). 

We utilize the networked improvement communities approach described by Bryk, Gomez, 
Grunow, and LeMahieu (2015) to increase the capacity for geometry instruction at the K-12 
level. This approach uses an organizational learning perspective in which all stakeholders are 
involved in the articulation of common problems as well as the design, monitoring, and 
continuous improvement of strategies to solve those problems. Bryk et al. (2015) indicate that 
achieving improvement at scale requires that particular attention be paid to variation within the 
system, and that understanding the sources of that variation is critical to achieving the goal of 
improvement research projects. 

Within the scope of our project, we are currently measuring variation by gathering and 
analyzing data in the following ways: 1) through an assessment instrument which tests GeT 
students’ mathematical knowledge for teaching geometry (MKT-G); 2) with end-of-term 
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questionnaires used to collect data on course content, student composition, and technology use; 
and 3) through the use of instructional logs, administered at three points throughout the term, 
aimed at gathering more nuanced information about instructional practices in GeT courses. In 
this paper, we describe these measures in more detail, and explore the potential use of the data 
gathered from these three instruments. 


Theoretical Framework 
Taking a Networked Improvement Communities Approach 

Bryk and colleagues (2015) set out a strategy for educational research and development 
called networked improvement communities (NICs), meant to harness the creative power of 
networked communities within an improvement science framework. One of the core principles 
of this approach is to “see the system that produces the current outcomes” (p. 57). The authors 
claim that oftentimes the traditional approach to solving complex educational problems is to 
quickly look for solutions to the problems without fully understanding the complex systems 
producing such problems. Russell and colleagues’ (2017) lay out a framework for the 
implementation of networked improvement communities and claim that education researchers 
and practitioners often struggle to find effective solutions as “[the] field is not organized to learn 
systematically, accumulate, and disseminate the practical knowledge needed for the 
improvement of teaching and learning” (p. 1). A second core principle of the NIC approach is to 
“focus on variation in performance” (Bryk et al., 2015, p. 13), with an acknowledgement that 
complex systems often result in considerable variation. This focus on variation reduces the 
tendency to oversimplify by looking for universal solutions to the problem, and supports more 
realistic consideration of which solutions work, for whom, and under what set of conditions. 

To study variation in performance, NICs must make use of consistent and practical 
measurement tools to inform the improvement efforts made by those involved (Morris & 
Hiebert, 2011). Additionally, a network hub is responsible for aggregating the data from those 
measures, as well as key insights that emerge, and feeding it back to the network so that 
innovations can be tested and integrated into new contexts (Bryk et al., 2015). We took on the 
role of network hub by creating a community of stakeholders and developing measures that 
would inform the community about progress toward a solution. As part of our work in this role, 
we facilitate opportunities for the network to engage with each other as well as the data being 
generated. 

Variation in GeT Courses 

Grover and Connor (2000) conducted a survey of 108 randomly selected U.S. colleges and 
universities to study the content and instructional practices of geometry courses through analysis 
of questionnaire responses and selected syllabi. They found considerable variation in content 
(e.g., geometries covered and axiomatic approaches employed), pedagogy (e.g., lecture, group 
work, and alignment with NCTM Professional Teaching Standards), and assessment (e.g., in- 
class examinations, homework, and forms of alternative assessment). 

To see if this variation in GeT courses persists, we conducted an analysis of GeT course 
artifacts from 17 initial participants in this study (1.e., syllabi supplemented with course catalogs, 
interviews, and poster sessions). The results of our study of GeT courses were very well aligned 
with Grover and Connor’s (2000) results, indicating that there still exists wide variation in GeT 
courses. 
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Methods 

As the project is intended to help provide support for the instructors of university GeT 
courses, it is important to work collaboratively with them in order to help define the problem 
space. We began by locating institutions with teacher preparation programs—as we are 
interested in the undergraduate geometry course serving teachers, rather than geometry courses 
in general. Within those institutions, we looked in mathematics departments for geometry 
courses serving secondary mathematics pre-service teachers and identified instructors of those 
courses as the natural members of this community. We conducted a set of 19 initial interviews to 
gather their professional perspectives as instructors of GeT courses. We organized the interview 
data according to common challenges that many of the instructors noted as embedded in the 
work of teaching GeT courses. These common problems, or tensions, that many instructors 
identify in their own work we identify as inherent in the work of teaching college geometry, 
which are perhaps distinct from the set of problems or defects that an outsider might identify in 
the work of a GeT instructor (Herbst, Milewski, Ion, & Bleecker, 2018; Milewski et al., 2019). 

In June 2018, we held a two-day conference which gathered more than fifty various 
stakeholders involved in the teaching of the GeT course (GeT instructors, high school district 
leaders, high school teacher leaders, and education researchers). Using what we had learned from 
the interviews with GeT instructors about the tensions that come with teaching the GeT course, 
we worked with conference attendees with a common problem in mind: to improve capacity for 
teaching K-12 geometry. Two of the tensions most salient in the conversations of the conference 
working groups were: 


e Knowledge tension: This tension arises in contexts where GeT instructors need to 
consider the question “What is the knowledge students need to learn in the GeT course?” 
On one side of the tension, the course exists to provide novice teachers with the 
knowledge needed to teach the secondary geometry course. This includes knowledge 
like, “What are the common ways that students think about parallelograms?” or “How 
does one design good proof problems for high school geometry students?” On the other 
side, GeT is a university mathematics course that needs to be comparable in terms of 
rigor with other advanced mathematical coursework that students are expected engage 
with. Ideally, the course could include both kinds of mathematical knowledge, however, 
time is a limited resource and GeT instructors need to decide what to prioritize. 

e Experiences tension: This tension arises in contexts where GeT instructors need to 
consider the question “What experiences can support students’ learning in the GeT 
course?” On one side of the tension, as GeT is a mathematics course, it would be 
reasonable to assume that students in the course are learning to think and act like 
mathematicians. Thus, students should experience opportunities to engage in reasoning, 
problem-solving, and other mathematical practices, which could serve them as future 
teachers as it would engage them in work similar to what they would have their eventual 
students engage. On the other side, the course acts a service course for pre-service 
teachers who are apprenticing into the work of a teacher, and this would suggest that they 
should experience opportunities to practice the work of teaching, like explaining a 
mathematical concept to the whole class or to each other, or interpreting fellow students’ 
work. 
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Rallying around these tensions, the conference participants collaborated to establish a set of 
projects to engage with over the course of the next four years that would address these tensions. 
Through the use of an online collaboration platform, teams of GeT instructors unite to work on a 
common project. An example of one of these projects is the Geometry Knowledge Needed for 
Teaching group, where 17 GeT instructors are currently working together to answer the 
questions: What is the mathematical content knowledge needed for teaching high school 
geometry? What do we need to teach our college geometry students so they begin their teaching 
careers with the appropriate knowledge? To answer these questions and discuss the tensions that 
come up in the work of teaching the GeT course as highlighted above, this group meets over 
video conference once a month, as well as throughout the month asynchronously, on an online 
forum. 

In addition to collaborating on these projects, GeT instructors also volunteer to take and 
administer various measurement instruments throughout the year. The details of these 
instruments are outlined in the following section. We understand the potential pitfalls of large 
variability in those courses at the same time that we honor the academic freedom and 
professional judgment of instructors. The project forums, like the one described above, are meant 
to promote improvement by creating community. To inform the improvement efforts made by 
instructors who participate in the community, we track variation in course offerings and variation 
in outcomes (in terms of MKT-G), with the hope to ascertain whether changes in the former are 
related to changes in the latter. If reduction in course variability could be correlated with 
increases in MKT-G gains, this would lend credibility to the notion that reducing variability in 
the course offerings would be desirable. 


Measures 
Mathematical Knowledge for Teaching Geometry (MKT-G) 

Herbst and Kosko (2014) have developed and validated an instrument used to measure MKT- 
G that targets four of the six domains of content knowledge for teaching identified by Ball, 
Thames, and Phelps (2008). These items are closed-ended and graded as either correct or 
incorrect. In a prior study, the instrument was administered to 387 inservice teachers from 47 
states, as well as 195 preservice teachers from 7 institutions. Results from these tests showed that 
preservice teachers’ scores on the MKT-G after taking a GeT course were .23 standard 
deviations lower than inservice teachers’ MKT-G scores. While this result was unsurprising, it 
raised the question of whether the test could detect differences in MKT-G gained over the course 
of a semester of GeT instruction. 

Over the course of the project, we will be collecting data from GeT students who will take 
this instrument at the beginning and end of their GeT course. Progress toward readiness to teach 
high school geometry will be measured in terms of average change in scores between pre- and 
post- assessments. Throughout the term, we share three sets of reports with participating GeT 
instructors: 1) individualized reports showing how their students performed on the pre-test 
compared to the national sample of practicing teachers; 2) individualized reports showing how 
their students performed on both the pre- and post- tests, with change in scores over the term 
reported in relation to the national sample of inservice teachers; and 3) aggregated reports 
showing each instructor how students from their institution performed on average compared to 
the overall average scores of all GeT students in this study. Our hope is that the instructors will 
use the individualized and aggregate results from these reports to reflect on and inform their 
course design and teaching practices, as well as spark conversations within the network about 
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rationales, costs, and benefits of any potential changes in practice with respect to the knowledge 
and experiences tensions identified in their work. 
Course Questionnaire and Instructional Logs 

Inspired by the work of Grover and Connor (2000), we have developed a course 
questionnaire to be taken by GeT instructors at the end of each term in which the course is 
offered. In the questionnaire, instructors are asked about the course sequence at their institution, 
student composition, textbook use, dynamic geometry software use, as well as types of geometry 
taught in relation to various axiomatic approaches. Some examples of questions include: 


e Approximately what percentage of the students enrolled in the geometry course are 
prospective high school or middle school teachers? 

e [What] courses are students required to take before taking this section of geometry? 

e List the most important Euclidean geometry theorem(s) you cover. 


To better understand the experiences students are having in GeT courses, we have developed 
a set of instructional logs to be completed by GeT instructors three times throughout the term in 
which the course is offered. In each of these logs, instructors are asked to report on the planning 
and implementation of two lessons. As part of each log, instructors are asked to share the set of 
expectations for the lesson, as well as the amount of class time they devoted to each expectation. 
Following this, they are asked about the instructional approaches employed throughout the 
lesson, reporting how much class time was spent on each (e.g., whole class lecture, student 
presentations). These set of logs serve as one way of describing the experiences students are 
having in the GeT courses. 

We expect these results to be useful in gauging variability across GeT courses, with the 
hypothesis that variability will be reduced as instructors participate in the networked 
improvement community. One reason that variability may be reduced over time is that GeT 
instructors will have access to these aggregated reports, may see evidence of other instructional 
practices being employed within the community, and will participate in discussions of the data 
all together. 


Preliminary Results and Discussion 

Fall 2018 MKT-G Results 

In Fall 2018, 94 students from six GeT courses completed the pre- and post- MKT-G 
assessment. The level of MKT-G possessed by each student was provided as IRT scores derived 
from the IRT (item-response theory) model which takes account of the difficulty and 
discrimination of items (how well each item can discriminate participants’ ability). In Table 1, 
we report on the pre- and post-IRT scores of the 94 GeT students, as well as the IRT gains from 
this sample. An average score of 0 for the sample of GeT students would indicate a level of 
knowledge equivalent to the average amount of knowledge of a national sample of in-service 
teachers (N=605) previously collected. The negative IRT scores reported indicate that GeT 
students’ knowledge is below the average amount of MKT-G of the in-service teacher sample 
(Ayala, 2009; Crocker & Algina, 2006). The positive gain in IRT scores (0.15) between the pre- 
and post-test indicates that GeT students made progress over the course of the term, on average, 
with respect to MKT-G, and the gain was significant (t(93)=-2.85, p=0.0027). The use of the 
data from practicing teachers allows us to understand the GeT students’ relative standing to the 
practicing teachers. 
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Table 1: Pre- and Post- test Score Comparison of GeT Students (N=94) and ISTs (N=605) 


IRT 0 
PRE_IRT POST_IRT GAIN_IRT 
GeT students (N=94) -1.29 -1.13 0.15 
ISTs (N=605) 0.00 0.00 N/A 


Despite overall positive gains by students on average, only four of the six GeT instructors 
showed positive gains in their students’ IRT scores. The graph below (Figure 1) represents the 
standing of the GeT students’ IRT scores in pre-test and post-test within the distribution of IRT 
scores from the national sample of in-service teachers. The GeT students’ post-test scores are 
shifted slightly to the right from their pre-test scores, indicating improvement in MKT-G over 
the semester of GeT instruction. Compared to the ISTs, most of the 94 students have lower than 
average level of MKT-G in both pre- and post- test scores and this result is consistent with the 
prior study. 
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Figure 1: Histogram of the MKT-G Scores of GeT Students (N=94) and ISTs (N=605). 


Course Questionnaire and Logs Results 

In our first distribution of the course questionnaire, while we have a small sample of 
instructors (N=7), we are able to make similar observations as Grover and Connor (2000) with 
respect to large variability in the GeT courses. Some sample responses are shown in Table 2. 


Table 2: Descriptive Results from Two Course Questionnaire Prompts (N=7) 


<=50% S1%- >81% 
80% 
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Q9: Approximately what percentage of the students enrolled in the 
geometry course are prospective high school or middle school n=2 n=2 n=3 
teachers? (N=7) 


Q31: Indicate what percentage of the course is devoted to 
Euclidean geometry. (N=7) 


Also, in the Fall 2018 term, we collected a total of 20 instructional logs from five GeT 
instructors. An example of results from one such question is shown in Figure 2. 


In this lesson, how many times did you ask students to make conjectures? (n=20) 


More than five times 


| didn't 10 (50.0%) Once or twice 


3 (15.0%) 


Three to five times 


Figure 2: Aggregated Responses (N=20) to One of the Instructional Log Questions 


The results from the course questionnaire and instructional logs have the potential to provide 
evidence of ways in which the knowledge and experiences tensions are present in the work of 
teaching the GeT course. We may find that GeT instructors with student populations that are less 
than 50% PSTs are less likely to spend considerable time on Euclidean Geometry, for example. 
Other uses for the data generated by these instruments is to measure variability across courses 
over time. The results shared here are from the first instructional term of this study, but we 
expect to look at how courses change over time when that data becomes available. For the time 
being, the instruments described here seem promising to allow us to describe the variability 
present across offerings of the GeT course. 
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